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1 Scope 

1.1 Identification 

This ‘‘Quarterly Progress Report” describes progress for AER Project 778 (P778) on “Distortion 
Representation of Forecast Errors for Model Skill Assessment and Objective Analysis” funded by 
NASA (NAS5-99088). 

The current reporting period (CRP) is 2 June 2001 - 1 September 2001. 

1.2 Project Overview 

The primary objective of P778 is a novel characterization of errors for numerical weather predictions. 
A general adjustment representation allows for the displacement and amplification or bias correction 
of forecast anomalies. 

Characterizing and decomposing forecast error in this way has several important applications, 
including the model assessment application, and the objective analysis application. In the cur- 
rent project, and its predecessor (contract NAS5-32953), we have focused on model assessment, 
restricted to a realistic but univariate 2 dimensional situation. In the previous contract we studied 
the forecast errors of the sea level pressure (SLP), the 500 hPa geopotential height, and the 315 K 
potential vorticity fields for forecasts of the short and medium range. The forecasts were generated 
by the GEOS (Goddard Earth Observing System) data assimilation system with and without ERS 

1 scatterometer data. This work is a first step towards (1) a testbed for the use of the adjustment 
representation of forecast errors, (2) a means of validating the GEOS data assimilation system and 
(3) a description of the impact of the ERS-1 scatterometer data. 

In the current project (contract NAS5-99088) we have concentrated on the application of the 
FCA to the ECMWF Lorenz data sets of 500 hPa height. 

1.3 Document overview 

This document records the development of the project. Progress during the CRP, as well as any 
problems encountered are detailed. This document is updated as needed and baselined every 
quarter. 

2 Current Progress 

2.1 Tasks 

2.1.1 Smoothness constraint based on Lorenz data set 

We completed the formulation of the smoothness penalty functional this past quarter. We used a 
simplified procedure (described below) for estimating the statistics of the FCA solution spectral 
coefficients from the results of the unconstrained, low- truncation FCA (stopping criterion) solutions. 

Our initial tests had shown that the FCA solution spectral coefficients, for a given trunca- 
tion wavenumber, are quite insensitive to the smoothness penalty function at low wavenumbers. 
Therefore, it is more important to properly model the observed FCA solution statistics at high 
rather than low wavenumbers. In our initial formulations, the smoothness weights were derived 
from optimizing the fit, to the rms spectral coefficient magnitudes, which tended to emphasize the 



spectral coefficient with large magnitudes at low wavenumbers. Applying a fitting procedure to 
the corresponding weights instead (which are proportional to the inverse of the square magnitude) 
w'as found to be computationally unstable, since the values at high wavenumbers are derived from 
a small sample of cases. 

We explored an alternative approach, in which the FCA solutions from an unconstrained, high- 
truncation run are used to define the shape of the smoothness weights in spectral space, but the 
magnitudes are scaled by the statistics from the unconstrained, truncated solutions. The scaling 
factor was allowed to vary with total wavenumber n. We encountered several problems in this 
approach: 

• using a single shape function for all forecast lengths resulted in FCA solutions that explained 
more (less) of the error variance than the corresponding stopping criterion solutions for long 
(short) forecast lengths. 

• attempts at using different rescalings for different forecast lengths resulted in an unacceptably 
large number of tunable parameters. 

Based on the results of our tests so far, and the observation that increasing the truncation 
wavenumber changed the FCA solution even when the magnitude of high wavenumber spectral 
coefficients was kept small by the smoothness constraint, it was decided to explicitly enforce a 
truncation (dependent on foreast lead time) selected by the stopping criterion in the majority of 
cases. Smoothness penalty function weights were determined separately for each season, forecast 
length, and FCA component. The weights were set equal to the inverse mean square value of the 
spectral coefficient magnitudes, with the following exceptions: 

• the truncation wavenumber was set so that there were at least 4 cases with truncations as 
large or larger out of the 30 stopping criterion FCA runs. 

• for wavenumbers n with 6 or fewer cases of nonzero spectral coefficients, an average rms was 
computed for all zonal wavenumbers m for that value of n. 

The relative weighting of the observation and penalty function components of the objective was 
selected initially such that both terms contribute roughly equal amounts to the objective function. 

The penalty function was formulated and applied for both the “full” statistics (the statistics 
and the penalty function are based on the absolute value of the FCA spectral coefficients), and the 
“difference” statistics (the difference of the FCA spectral coefficients between forecasts of increasing 
forecast length). 

An examination of the results shows that overall, the fraction of the forecast error variance 
explained by the penalty function FCA solutions is in general agreement with that of the corre- 
sponding stopping criterion solutions. This result holds equally true at all forecast lengths. A 
scatterplot of penalty function vs. stopping criterion fraction of explained variance (aggregated 
over all forecast lengths, see Fig. 1) show's three distinct clusters of points: 

• cases for which the stopping criterion truncation was less than that of the penalty function 
FCA. Almost of these points are above the diagonal. 

• cases for which the stopping criterion truncation was equal to that of the penalty function 
FCA. These points are closer to the diagonal, but predominantly below it. 
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Figure 1: Comparison of constrained (penalty function) and unconstrained (stopping criterion) 
FCA solution fraction of explained forecast error variance. Results are aggregated for all cases and 
all forecast lengths for one season. 






• very few cases for which the stopping criterion truncation was greater than that of the penalty 
function FCA (recall that there can be at most 3 cases per forecast length). These points are 
all below the diagonal. 

These results indicate that the fraction of the explained variance for the constrained (penalty 
function) FCA solutions is primarily governed by the specified truncation wavenumber; the smooth- 
ness penalty function does, however, further constrain the solution, resulting in average error statis- 
tics of the correct magnitude. 

A comparison of the spectral coefficient magnitudes shows that the constrained solutions have 
generally smaller FCA spectral coefficients, particularly at low wavenumbers, and for the displace- 
ment FCA components (see Fig. 2 and 3). 

2.1.2 Water Vapor Imagery 

During the current reporting period we have completed the calculation of GEOS-2 model-equivalent 
brightness temperatures for the 6.7 micron and 11 micron window channels used in the GOES 
imagery for all 10 cases from August 1999. These were simulated using the AER-developed Optimal 
Spectral Sampling (OSS) model. We are currently finalizing the data preparation and minimization 
software for FCA application on the observed and simulated imagery. 

2.2 Resources 

2.2.1 Hardware and Software 

No equipment or property was acquired in the CRP. 

2.2.2 Personnel 

There were no changes in project personnel during the CRP. 

2.2.3 Travel 

There was no travel during the CRP. 

2.2.4 Government Furnished Equipment (GFE) 


None. 

2.3 Risks 

Currently there are no outstanding problems or risks. During the CRP period no new risks were 
identified, 

2.3.1 Difficulties Encountered 


None. 
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Figure 3: Comparison of constrained (penalty function) and unconstrained (stopping criterion) 
FCA solution spectral coefficient magnitudes for difference statistics- 
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Figure 2: Comparison of constrained (penalty function) and unconstrained (stopping criterion) 
FCA solution spectral coefficient magnitudes for full statistics. 





2.3.2 Research That Has Failed 

None. 


2.4 Current Plans 

During the next reporting period we will apply the derived penalty functional to the entire Lorenz 
dataset, and analyze the results. We will also begin baseline testing of the FCA algorithm using 
the observed and simulated datasets described above. 

2.5 Publications 

There were no publications during the CRP. 

2.6 Staff Organization 

The project is a project of the Numerical Weather Prediction group at AER. The Principal Inves- 
tigator, Dr. R. N. Hoffman is responsible for the overall technical tasks and their integration. 

2.7 Fiscal Information 

2.7.1 Tracking and Control Mechanisms 

Fiscal tracking and control mechanisms for the project are provided by the AER contract manage- 
ment and administration team headed by Ms. Cecilia Sze, Chief Executive Officer. On a monthly 
basis, this team prepares a report of actual costs incurred by the project and, after a review with 
the Principal Investigators, makes adjustments to the Estimate at Completion Report as necessary. 
The monthly review is also the mechanism established by AER for management to monitor the 
progress of the contract and to provide whatever assistance is needed by the Principal Investigators 
to meet schedule and other project objectives. The management team is responsible for preparing 
and submitting the required cost data. Cost data and summary fiscal information are provided 
separately on a monthly basis. 
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